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Need. 

From  its  very  inception,  it  was  envisioned  that  linear  programming 
would  be  applied  to  very  large,  detailed  models  of  economic  and  logistical 
systems  [Wood  and  Dantzig  (1947)].  Kantorovich's  1939  proposals,  which  were 
before  the  advent  of  the  electronic  computer,  mentioned  such  possibil¬ 
ities.  In  the  intervening  25  or  so  years,  electronic  computers  have 
become  increasingly  more  powerful,  permitting  general  techniques  for 
solving  linear  programs  to  be  applied  to  larger  and  larger  practical 
problems.  In  the  author's  opinion,  however,  additional  steps  are  necessary 
if  there  is  to  be  significant  progress  in  solving  certain  critical 
problems  that  face  the  world  today. 

The  conference  on  Large-Scale  Resource  Allocation  Problems 
held  at  Elsinore,  Denmark,  July  5-9,  1971  represents  an  historic  first 
because  it  demonstrates  that  optimization  of  very  large-scale  planning 
problems  can  be  achieved  on  significant  problems.  We  cite  some  examples 

1 

from  the  conference: 

Arthur  Geoffrion's  paper  "Optimal  Distribution  System  Design" 
is  of  interest  because  (1)  it  described  the  suc<essful  solution  of  a 
large-scale  problem  from  commerce,  (2)  it  involved  discrete  variables 

(representing  the  integer  number  of  warehouses  to  be  buiit  or  closed 
s - 

Parts  of  the  material  in  this  paper  where  drawn  from  a  draft  of  a 
proposal  to  establish  such  a  laboratory  at  Stanford  University  prepared 
by  R.  Cottle,  B.  C.  Eaves,  G.  H.  Golub,  F.  S.  Hillier,  A.  S.  Manne, 

D.  J.  Wilde,  R.  B.  Wilson  and  George  B.  Dantzig. 
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down),  (3)  it  successfully  combined  a  variety  of  advanced  techniques 
in  a  single  computer  program. 

Leon  Lasdon's  paper  "Uses  of  Generalized  Upper  Bounding 
Methods  in  Production  Scheduling"  is  of  interest  because  it  not  only 
(1)  described  a  successful  large-scale  application  (this  time  to  a 
rubber  factory),  and  (2)  made  use  of  advanced  techniques,  but  also, 

(3)  because  it  showed  the  possibility  of  automatically  scheduling 
day-to-day  operations  consistent  with  the  long-terms  goals,  i.e.,  it 
successfully  combined  short  and  long-term  planning  goals  of  an 
enterprise. 

The  papers  by  several  authors  (for  example  those  of  Abadie, 

Buzby,  Heard)  are  particularly  noteworthy  because  they  described  the 
successful  solution  of  real  problems  (Electric  Energy  Production  and 
Olefin  Production)  that  were  essentially  non-linear  and  large-scale 
in  nature. 

Society  will  benefit  greatly  if  certain  total  systems  can 
be  modeled  and  successfully  solved.  For  example,  crude  economic 
planning  models  of  many  developing  countries  indicate  a  potential 
growth  rate  of  GNP  of  10%  to  15%  per  year.  Implementation  of  such  a  growth 
(aside  from  political  difficulties)  requires  a  detailed  and  carefully 
worked  out  model  and  the  availability  of  computer  programs  that  can 
solve  the  resulting  large-scale  systems.  The  world  is  currently  faced 
with  difficultproblems  related  to  population  growth,  availability  of 
natural  resources,  ecological  evaluation  and  control,  urban  redesign, 
design  of  large-scale  engineering  systems  (e.g.,  atomic  energy  and 


i 


I 

-  3  - 

recycling  systems),  and  the  modeling  of  man's  physiological  system 
for  the  purpose  of  diagnosis  and  treatment.  These  problems  are  complex, 
are  urgent. and  can  be  solved  only  if  viewed  as  total  systems.  If  not, 
then  only  patchwork,  piecemeal  solutions  will  be  developed  (as  it  has 
been  in  the  past)  and  the  world  will  continue  to  be  plagued  by  one 
crisis  after  another  caused  by  poor  planning  techniques.  For  solutions, 
these  problems  require  total  system  planning,  modeling  and  optimization. 

It  is  my  belief  that  it  is  necessary  at  this  time  to  create 
several  system  optimization  laboratories  where  enough  critical  mass 
would  exist  that  representative  large-scale  models  (of  the  type 
referred  to  above)  could  be  practically  modeled  and  numerically  solved. 

Solving  large-scale  systems  cannot  be  approached  piecemeal  or  by 
publishing  a  few  theoretical  papers.  It  is  a  complex  art  requiring 
the  development  of  a  whole  arsenal  of  special  tools. 


Background. 

The  optimization  of  large-scale  systems  is  technically  an 
extremely  difficult  subject.  Historically,  starting  with  U.S.  Air  Force 
problems  in  1947,  linear  programs  were  formulated  to  solve  just  such 
systems.  These  problems  involved  systems  of  interlocking  relations 
involv  j  many  planning  periods,  combat  units,  types  of  personnel 
and  supply.  It  lead  to  thousands  of  equations  in  many  thousands  of 

unknowns,  and  hence  was  beyond  computational  capabilities.  It  became  necessary 
to  severely  restrict  the  class  of  practical  problems  to  be  solved. 


Starting  around  1954  a  series  of  purely  theoretical  papers  began  to 
appear  on  how  to  efficiently  solve  large  systems  and  by  1970  they 
numbered  about  200.  There  was  little  in  the  nay  of  implementation. 

Exceptions  were  the  out-of-kilter  algorithms  for  network  flow  problems 
proposed  by  Ford  and  Fulkerson  [1958]  and  the  "decomposition  principle" 
of  Philip  Wolfe  and  myself  which  had  been  tried  but  with  variable 
results  [I960].  On  the  other  hand  a  more  modest  proposal  of  Richard 
Van  Slyke  and  myself  (general  ized-upper  bounds)  has  been  very 
successful  [1967].  Apparently  a  great  deal  in  the  way  of  impirically 
testing  of  ideas  is  necessary.  Such  testing,  however,  has  not  been  easy 
to  do  because  pertinent  test  models  are  complex  and  large  and  cost  a  great  deal 
of  money  to  program  and  solve.  Therefore  progress  has  been  slow  up  to  the 
time  of  the  Elsinore  meeting. 

Since  the  origins  of  mathematical  programming  in  the  development  of 
transport  allocation  methods  in  the  early  1940's,  and  especially  since  the 
introduction  of  the  Simplex  Method  of  linear  programming  in  1947,  the 
power  of  the  solution  methods  as  well  as  the  size  and  type 
of  applications  have  grown  enormously.  In  the  intervening 
decades  the  methodology  has  been  extended  to  include  non-linear  and 
integer  programming,  dynamic  programming  and  optimal  control,  and  a 
host  of  other  types  of  optimization  problems.  The  range  of  applications 
has  been  extended  from  simple  allocation  problems  to  an  enormous  variety 
of  problems  in  intertemporal  allocation  and  investment  planning, 
engineering  design  and  optimization,  and  scientific  studies  or  physical, 
biological,  and  ecological  systems.  There  is,  in  fact,  no  end  foreseeable 
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to  the  applications  of  mathematical  programming  to  a  number  of 
important  (and  crucial)  optimization  problems. 


Some  Examples  of  Important  Applications. 

A.  INVESTMENT  PLANNING  (INTERTEMPORAL  ALLOCATION):  Problems 
of  aggregate  economic  planning  for  a  (developing)  country,  as  noted  earlier, 
present  an  exploitable  special  structure  that  has  been  studied  intensively  and 
has  great  potential.  Related  structures  occur  in  problems  of  dynamic 
programing  and  optimal  control.  Related  but  more  complicated 
structures  arise,  for  example,  in  problems  of  plant  location  and 
time-phasing, and  in  Investment  planning  in  general  in  the  firm. 


8.  DECENTRALIZED  ALLOCATION:  Modern  methods  of  decomposition,  are 

based  on  the  class  of  decentralized  allocation  problems  in  which  scarce 
resources  are  to  be  allocated  among  several  otherwise  independent 
enterprises  or  "divisions".  Closely  related  is  the  class  of  problems 
of  two-stage  allocation  under  uncertainty,  for  which  it  is  know  in 
the  linear  case  that  the  dual  problem  is  one  of  decentralized 
allocation.  It  is  of  particular  importance  to  realize  that  the 
"divisional  subproblems"  may  themselves  be  of  a  special  structure 
(e.g.,  a  transportation  problem)  which  can  be  exploited. 

C.  ENGINEERING  DESIGN  AND  OPTIMIZATION:  A  variety  of 
engineering  design  and  process  optimization  problems  present  specially- 
structured  mathematical  programs  whose  characteristic  features 
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are  highly  dependent  on  the  process  being  studied.  Problems  of  this 
type  illustrate  the  need  for  a  flexible  and  comprehensive  software 
package  from  which  components  can  be  drawn  to  build  up  models  of  very 
complex  systems. 

D.  PHYSICAL,  BIOLOGICAL,  AND  ECOLOGICAL  SYSTEMS:  A  number 
of  prob'ems  in  the  physical  sciences  (e.g.,  X-ray  crystallography) 
and  biological  sciences  (e.g.,  models  of  body  processes)  present 
specially-structured  mathematical  programing  problems.  A  good 
example  are  models  cf  ecological  systems  in  which  the  many  and  varied 
relationships  among  the  components  again  require  a  flexible  and 
comprehensive  software  package. 

E.  URBAN  PLANNING:  Coordinated  planning  of  the  many  component 
subsystems  (e.g.,  transport,  recreation,  education,  etc.)  of  an  urban 
environment  presents  a  complex  kystems  optimization  problem  for  which 
ordinarily  the  most  powerful  and  flexible  methods  are  required. 

F.  LOGISTICS:  Coordinated  logistics  support  or  for  any 
large  industrial  (e.g.,  warehousing  and  transport)  or  government 
(military)  activity  normally  presents  a  systems  optimization  problem 
of  considerable  size  and  complexity,  buc  with  exploitable  structural 


features. 


G.  TRANSPORTATION  SYSTEMS:  Various  problems  concerning  the 


design  of  transportation  systems  can  bo  formulated  as  network  optimization 
models  of  a  combinatorial  nature.  These  models  typically  have  very 
special  mathematical -programming  structures  for  which  highly  efficient 
algorithms  can  be  devised. 


The  Functions  of  a  Systems  Optimization  Laboratory. 

The  purpose  of  such  a  laboratory  would  be  to  support  the 
development  of  computational  methods  and  associated  computer  routines 
for  numerical  analysis  and  optimization  of  large-scale  systems.  The 
ultimate  objective  of  the  development  effort  would  be  to  provide  an 
integrated  set  of  computer  routines  for  systems  optimization  which: 

I.  is  freely  and  publicly  available  to  users  of  government, 
science,  and  industry, 

II.  is  thoroughly  practical  and  widely  useful  in  applications  to 
diverse  kinds  of  Urge-scale  systems  optimization  problems, 

III.  embodies  the  most  powerful  techniques  of  mathematical 
programming  and  numerical  analysis,  and 

IV.  has  been  thoroughly  tested  for  efficiency  and  effectiveness 
on  representative  systems  optimization  problems  arising  in 
practice. 

The  Development  effort  of  such  a  laboratory  in  its  initial  stages 
would  consist  of  three  basic  activities:  [1]  research  in  mathematical 
programming,  including  particularly  the  anslysis,  development,  and 
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testing  of  special  computational  methods  for  certain  specially 
structured  optimization  problems  that  occur  frequently  in  systems 
optimization,  or  as  subproblems  in  larger  systems  optimizations, 

[2]  collection  of  representative  systems  optimization  problems  arising 
in  practice  in  government,  science,  and  industry,  in  order  both  to 
study  their  mathematical  structure  and  to  use  as  test  problems  for 
studies  of  efficiency;  and  [3]  development  of  an  integrated  set  of 
computer  routines,  and  an  associated  macro- language  to  enable  its 
flexible  use,  which  implements  the  most  powerful  of  existing  methods 
for  systems  optimization. 

Nature  of  the  Bottleneck. 

The  creation  of  such  a  laboratory  would  be  a  concerted 
effort  to  break  a  bottleneck  which  is  currently  constricting  the 
applications  of  mathematical  programing  to  many  of  the  most  important 
systems  optimization  problems.  There  is  a  lack  of  an 
integrated  collection  of  compatible  computer  routines,  preferably 
organized  and  callable  via  a  macro-language,  which  can  be  employed 

efficiently  and  flexibly  in  a  wide  diversity  of  practical  applications. 

The  existing  methods  of  mathematical  programming  exploit 

either  general  structure  or  very  special  structure.  Those  that  exploit 
general  structure  take  advantage  of  the  fact  that  in  a  particular 
problem,  the  functional  forms  involved  are  linear,  or  quadratic,  or 
convex,  separable,  etc.  Methods  of  this  kind  ordinarily  are  limited 
in  their  applications  by  the  size  and  speed  of  the  computing  equipment 
available  according  to  some  power  (often  the  third  or  fourth)  of  the 
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number  of  variables  and/or  constraints.  Those  that  exploit  a  very  special 
structure  take  advantc-^  of  further  particular  features  of  a  problem. 

For  example,  in  the  case  of  linear  programming,  which  is  the  most 
highly  developed  in  this  respect,  there  are  methods  which  exploit 
the  special  structures  of  (1)  network  problems  arising  in  transport 
planning,  (2)  "block-diagonal”  problems  arising  in  decentralized 
allocation  problems,  (3)  "staircase"  problems  arising  in  dynamic 
investment  planning,  economic  growth  models,  and  optimal  control, 

(4)  problems  amenable  to  "column  generation"  arising  in  production 
scheduling  and  elsewhere,  (5)  general  problems  with  "sparse"  matrices 
etc.  Moreover,  there  is  a  substantial  and  powerful  theory  of  how  to 
decompose  large  and  complicated  systems  into  their  component  subsystems 
and  from  analyses  of  these  components  to  derive  solutions  to  the 
original  large  system.  Methods  that  exploit  special  structure  are 
not  limited  in  the  range  of  their  applicability  in  the  way  that 
ordinary  general-structure  methods  are;  indeed,  with  present  methods 
and  computing  equipment  it  is  practical  in  certain  cases  to  solve 
systems  with  close  to  a  million  of  variables  and  constraints.  (For 
example  the  National  Biscuit  Company  problem  solved  by  Mathematica. ) 

Fortunately,  it  is  in  the  nature  of  human  activity,  and  in  large 
part  of  the  physical  world  as  well,  that  large  and  complicated  systems  can 
often  be  viewed  as  systematic  heirarchies  of  interrelated  subsystems. 
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These  special  structures  permit  numerical  analysis  and  optimization 
via  methods  cited  above,  whereas  general -structure  methods  as  noted 
would  be  infeasible  if  the  problem  were  of  the  size  normally  encour  lered 
in  practice. 


The  bottleneck,  however,  is  that  presently  there  is  not 
available  any  collection  of  decomposition  methods  and  special-structure 
methods  implemented  in  freely  available,  efficient,  tested,  flexible 
computer  routines  which  can  be  applied  easily,  cheaply,  and  with  con¬ 
fidence  to  practical  problems  as  they  arise.  The  result  has  been,  and 
will  continue  (if  development  work  does  not  proceed),  that  in  each 
potential  appl  ication  it  would  be  necessary  to  develop  computer  routines 
especially  for  the  project.  Because  this  is  so  costly  in  effort  and  time, 
such  development  is  generally  not  done  and  the  valuable  potential  application 
to  the  system  optimization  is  foregone. 

There  are  three  reasons  for  this  unfortunate  state  of  affairs. 

One  is  that  in  the  past  researchers  on  decomposition  methods  and 
special -structure  methods  have  not  had  a  viable  way  of  enabling  their 
work  to  contribute  directly  to  the  construction  of  such  a  collection 
of  computer  routines.  Either  there  was  no  incentive  to  complement 
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their  research  results  with  practically  useful  computer  routines;  or, 
if  they  did  do  it,  there  was  no  way  that  the  routines  could  be 
documented,  tested,  and  ultimately  incorporated  into  a  larger 
collection  of  established  routines;  indeed,  there  has  been  almost 
a  complete  absence  of  standard  documentation  procedures,  standard 
test  problems,  and  standard  compatibility  requirements  for  callabllity,  and 
data  input,  and  output.  The  consequence  has  been  that  research, 
implementation,  and  applications  of  systems  optimization  to  everyone's 
detriment  have  been  uncoordinated  and  disconnected. 

The  second  reason  is  that  the  incentives  for  development  work 
have  operated  at  cross  purposes  with  the  ultimate  goals.  As  mentioned, 
in  a  particular  application  it  is  usually  too  costly  or  time-consuming 
to  undertake  the  development  of  the  needed  computer  routines,  or  just 
as  likely,  the  organization  faced  with  the  tasks  lacks  the  expert 
competence  among  its  staff  to  complete  the  job  successfully.  On  the 
other  hand,  occasional  development  work  has  been  undertaken  by  private 
software  firms.  Indeed,  five  or  ten  years  ago  one  would  have  had 
great  hopes  that  this  approach  would  succeed.  In  fact,  however,  the 
incentive  to  private  firms  has  in  nearly  every  instance  been  to  keep 
their  routines  proprietary,  expensive  to  use,  and  noncallable.  For 
the  most  part,  private  firms  have  responded  to  the  natural  incentive 
to  appropriate  the  public  know-how  into  a  privately  saleable  commodity. 

The  third  reason  is  that  there  has  not  been  support  for  a 
coordinated  development  effort,  one  that  assembles  expert  competence 
in  theory,  numerical  methods,  and  computer  science,  and  that  ensures 
the  permanence  of  its  work  through  a  thorough  program  of  experimentation, 
testing,  documentation,  and  enforced  compatibility  requirements. 
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A  Systems  Optimization  Laboratory  could  be  carefully  designed 
to  overcome  these  impediments  to  progress  in  tlr  field.  It  could 
bring  together  the  various  kinds  of  expert  competence  that  are  needed, 
and  it  could  implement  the  development  effort  in  a  coordinated  program 
of  research,  programming,  experimentation,  testing,  and  documentation. 

The  results  of  such  a  laboratory  could  be  made  freely  and  widely 
available  for  diverse  applications  in  a  flexible  and  easily  used  form. 

The  research  of  a  System  Optimization  Laboratory  can 
be  classified  broadly  as  follows.  (1)  basic  studies  related  to 
optimization  theory,  (2)  development  of  computational  methodology 
for  mathematical  programming,  including  general-structure  methods, 
decomposition  methods,  and  special-structure  methods,  and  (3)  construction 
and  evaluation  of  algorithms. 


Software  Developme 

A  major  act  System  Optimization  Laboratory  would  be 

the  development  of  software  packages  for  systems  optimization.  This 
development  effort  could  proceed  on  two  different  levels.  The  first 
major  activity  would  be  the  completion  of  a  macro-language  for 
organizing  and  calling  routines  in  the  software  package.  This  activity  could 
be  mainly  an  extension  of  the  macro-language  Mathematical  Programming 
Language  [MPL]  under  development  by  the  author.  The  second  major 
activity  could  be  the  programing,  testing,  and  documentation  of 
algorithms  for  decomposition  and  special  structures,  including 
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experimentation  with  alternative  algorithms,  and  testing  of  algorithms 
on  practical  problems.  Computer  routines  would  be  thoroughly 
documented,  tested  on  standard  problems,  and  written  In  a  formate 
compatible  with  and  callable  by  the  macro-language. 

External  Affairs. 

Three  Important  activities  of  the  Laboratory  fall  under  this 
heading.  First,  members  of  the  technical  staff  could  undertake  the 
collection  aid  study  of  examples  of  systems  optimization  problems 
arising  in  government,  science,  and  Industry,  for  use  both  as  test 
problems  and  as  Indicators  of  the  types  of  systemsand  specially- 
structured  problems  of  major  importance  in  practice.  Many  examples 
are  already  known,  but  further  empirical  data  is  considered  desirable 
to  ensure  the  ultimate  usefulness  of  the  Laboratory's  work.  Second, 
other  researchers  in  the  field  could  be  solicited  to  obtain  algorithms, 
computer  routines,  and  test  problems  for  inclusion  In  the  Laboratory's 
studies.  Also,  the  Laboratory  could  disseminate  information  to 
potential  contributions  on  the  requirements  for  computer  routines  to 
be  compatible  with  the  Laboratory's  software  package.  Third,  when 
the  Laboratory's  software  package  is  reasonably  complete,  it  could 
undertake  to  make  it  available  to  users  --  this  being,  of  course, 
the  ultimate  purpose  of  the  Laboratory. 
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Some  Research  Projects  of  a  System  Optimization  Laboratory. 

A  major  goal  of  the  Systems  Optimization  Laboratory  would 
be  to  provide  standardized  computer  routines  for  systems  optimization. 
The  types  of  research  activities  that  would  be  needed  to  supoort  this 
effort  are  outlined  below.  Particular  areas  of  research  that  miyht 
be  planned  for  the  initial  project  period  will  be  described  first: 


A.  Decomposition  Methods:  The  chief  requirement  in  the  construction 
of  numerical  methods  for  optimizing  large  systems  is  that  the  algorithm 
exploit  the  special  structure  of  the  system.  The  body  of  theory  and 
techniques  which  addresses  this  requirement  and  decompose  or  partition 
the  problem  into  smaller  problems  are  generally  called  decomposition 
methods. 

One  preliminary  task  in  the  development  of  decomposition 
methods  would  be  the  construction  of  an  efficient  taxonomy  for  sy.stem 
structures.  This  task  is  only  partially  complete.  The  major  taxonomies 
features  that  are  well  understood  can  be  described  briefly  as  fellows. 
First,  there  is  a  large  and  important  class  of  problems  whose 
special  structure  permits  the  design  of  an  efficient  algorithm  based 
directly  on  their  structure.  Usually,  duality  and  compact  representation 
schemes  play  a  key  role  in  the  design  of  the  network  problems,  problems 
with  upper  and  lower  bound  constraints,  and  a  number  of  nonlinear  problems 
(geometric  programming,  fractional  programming,  variable-factor 
programming,  etc.).  Often  problems  with  these  special  structures 


occur  as  subproblems  in  larger  systems  and  it  is  therefore  important 
to  have  available  efficient,  tested,  and  documented  routines  for  these 
problems  which  are  easily  callable- 

A  second  major  class  of  problems  are  those  which,  in  the  linear 
case,  are  characterized  by  sparse  matrices.  (Hence  the  numerical 
structure  is  quite  general  except  for  the  known  presence  of  many  zeros.) 
Compact  representation  schemes  for  sparse  matrices  play  the  major  role 
in  the  development  of  algorithms  for  these  problems  [Dantzig  (1963)]. 

A  third  class  of  problems  are  those  which  are  amenable  to 
generation  techniques.  The  major  examples  from  this  class  are  the 
column  generating  techniques  of  Gilmore  and  Gomory  (1961,  1965)  for 
"cutting-stock"  and  related  problems,  and  the  row  and  column  generating 
techniques  of  Wilson  (1972)  for  2-person  games  in  extensive  forms,  both 
of  which  use  dynamic  programming  as  the  means  of  generating  data 
explicitly  that  is  otherwise  embodied  implicitly  in  the  problem 
formulation.  A  generating  technique  of  much  grea  ^ality  is  the 

method  of  generalized  programming  in  which  it  is  rt  d  only  that  the 
data  be  generated  from  a  convex  set  using  duality  information  from  a 
master  coordinating  problem. 

The  fourth  class  of  problems  are  linear  programs  with  block- 
angular  structures  [Dantzig  and  Wolfe  (1962)],  [Chapter  22  in  Dantzig 
(1963)]  (in  either  primal  or  dual  form,  including  multi-stage 
programming  under  uncertainty  [cf.  Dantzig  and  Madansky  (1961) 


and  a  variety  of  other  dynamic  programming  problems]  represent  a 
problem  of  allocation  scarce  resources  to  otherwise  Independent  sub¬ 
problems.  Zschau's  (1967)  primal  decomposition  algorithm  also  applies 
to  this  class  of  problems,  which  are  of  prime  Importance  In  applications. 

Wolfe's  generalized  programming  approach  is  also  applicable 
to  a  fifth  major  class  of  problems  which  is  closely  related  to  the 
previously  mentioned  class,  namely  the  class  of  multi-stage  allocation 
problems  represented  by  dynamic  Investment  problems  and  optimal 
control  problems.  Another  example  Is  the  linear  control  problem 
which  can  be  solved  using  generalized  programming  [see  Dantzlg  (1966)]. 

Both  of  these  last  two  classes  are  Instance  of  a  general 
class,  which  can  be  called  nearly  decomposable  problems.  In  this 
general  class  one  finds  a  macro-structure  which  would  be  perfectly 
decomposable  Into  Independent  subproblems  except  for  the  presence 
of  a  relatively  few  connections  (and  therefore  interdependencies) 
among  the  subproblems.  The  development  of  efficient  algorithms  for 
nearly  decomposable  problems  Is  a  major  area  for  research  and  one  for 
which  the  range  of  applications  Is  enormous.  Its  successful  conclusion 
may  require  the  development  of  general  methods  for  highly  connected 
systems,  such  as  have  been  recently  proposed  by  Douglass  Wilde 
(unpublished).  One  form  of  such  a  method  Is  presently  available  In 
Bender's  decomposition  method  (1962). 

Surveys  of  the  major  decomposition  methods  are  given  by 
Geoffrion  (1970)  and  Lasdon  (1971). 
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In  the  area  of  decomposition  methods,  the  Systems  Optimization 
Laboratory  would  pursue  essentially  three  research  and  development 
activities.  First,  a  major  effort  could  be  to  program, test, and 
document  existing  decomposition  methods  as  part  of  the  development  of  the 
macro-language  MPL  [Dantzig  et  al .  (1970)].  This  development  effort  is 
aimed  at  creating  a  useful  software  package  for  many  of  the  most 
important  systems  optimization  problems  which  arise  in  practice.  Second, 
a  part  of  the  research  effort  could  be  devotes  to  the  construction  of 
new  algorithms  for  general  nearly-decomposable  problems  and  for  tightly 
connected  systems. 

The  third  part  of  the  research  program  would  reflect  the 
important  role  of  structural  taxonomy  in  the  development  of  decomposition 
methods.  In  connection  with  the  Laboratory's  empirical  studies  of  some 
of  the  major  systems  optimization  problems  encountered  in  practice, 
a  structural  taxonr.ny  could  be  developed  and 
comparative  studies  made  of  the  relative  efficiencies  of  alternative 
methods  of  optimizing  systems  of  similar  structures.  There  are,  moreover, 
a  number  of  systems  optimization  problems  of  known  structure,  and  of 
great  practical  importance,  for  which  an  intensive  development  effort 
could  be  devoted  to  the  construction  of  efficient  algorithms.  First 
on  this  list  is  the  class  of  "staircase"  problems  represented  by 
dynamic  investment  models  in  economics  and  business  and  optimal  control 
problems  arising  in  (among  other  contexts)  ecological  models. 

In  general,  the  Laboratory's  work  on  decomposition  methods 
would  provide  a  synthesizing  focus  for  its  entire  spectrum  of  studies  on 
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systems  optimization.  The  primary  objective  would  be  to  provide  an 
unified  body  of  theory,  methods,  and  computer  routines  for  the  efficient 
and  practicial  numerical  analysis  of  large  systems. 

B.  Mathematical  Programing,  Matrix  Decomposition  and  Sparse 
Matrix  Techniques:  (The  comments  of  this  section  are  due  to  Gene 
Golub.)  For  many  algorithms  in  mathematical  programming  it  is 
necessary  to  compute  a  sequence  of  matrix  decompositions.  For  example, 
in  the  classical  simplex  algorithms  for  solving  linear  programming 
problems  it  is  necessary  to  solve  two  or  th^ee  systems  of  linear  algebraic 
equations  at  each  iteration.  Thre  are  many  ways  of  solving  these  systems, 
but  a  particularly  effective  numerical  algorithm  is  to  use  some  form  of  the  LU 
(Lower-Upper  triangularization)  decomposition  of  a  matrix.  At  each  stage  of 

the  simplex  algorithm  the  coefficient  matrix  is  changed  by  r*]e  column  so  that  one 
is  concerned  with  techniques  of  updating  the  matrix  decomposition  in 

an  efficient  and  stable  manner,  especially  when  the  data  matrix  is  very 
sparse. 

In  general,  suppose  that  a  matrix  A  and  some  factorization 
of  A  ar?  given,  e.g.,  A  =  PTq\  where  P  and  Q  are  orthogonal 
matrices  and  T  is  a  triangular  matrix.  The  problem  then  is  to 
compute  the  factorization  of  A  +  ouvj  where  u,  and  v^  are  given 
vectors  and  a  is  a  scalar  quantity,  or  a  factorization  of  A  when 
A  is  changed  by  one  column. 

The  three  basic  considerations  in  computing  the  new  factorization 
are  the  following:  (1)  The  updating  should  be  performed  in  as  few 
operations  as  possible.  This  is  especially  true  when  handling  large 
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masses  of  data  where  continual  updating  is  needed.  (2)  The  numerical 
procedure  should  be  stable.  Some  procedures  which  have  been  recommended 
in  the  literature  can  easily  become  numerically  unstable.  This  is 
especially  true  for  the  Cholesky  factorization  of  a  matrix  when  o  =  -1 . 
(3)  The  updating  procedure  should  preserve  sparsity.  Quite  often  the 
original  matrix  factorization  will  be  sparse,  and  it  is  desirable  to 
preserve  the  sparsity  by  possibly  rearranging  the  rows  and  columns  of 
the  original  data  matrix. 

The  problem  of  updating  occurs  in  many  other  contexts,  e.g., 
statistics  and  control  theory.  For  this  reason,  it  is  especially 
important  to  have  methods  which  yield  fast,  accurate,  and  sparse 
factorizations. 

Therefore,  a  study  would  be  made  of  various  factorizations  and 
how  they  may  be  used  in  large-scale  programming  problems,  especially 
when  the  data  matrix  is  structured.  The  sparse-matrix  techniques 
are  especially  useful  as  an  alternative  when  the  decomposition 
principle  is  applicable.  Furthermore,  the  matrix-decomposition 
methods  would  be  most  useful  when  the  complementarity  methods  for 
solving  mathematical  programming  problems  are  applicable.  Some  study 
has  already  been  made  in  this  direction  [Tomlin  (1971)]. 
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C.  Complementarity  Methods:  The  development  of  complementarity 
methods  is  the  major  advance  in  the  theory  and  technique  of  mathematical 
programming  in  recent  years.  The  application  of  this  approach  to 
decomposition  and  special-structure  methods  remains  largely  undeveloped, 
however,  There  is  a  prospect,  moreover,  that  further  development  of 
the  present  general -structure  complementarity  methods  will  lead  to 
substantial  improvements  in  their  efficiency  and  range  of  applications. 

Due  to  the  probably  importance  of  complementarity  methods  in  the 
development  of  new  algorithms,  the  research  program  could  pursue  several 
major  topics  in  this  area. 

[1]  Linear  Complementarity.  Linear  complementarity  problems 
arise  in  linear  and  quadratic  programming  and  in  2-person  games,  and 
they  are  a  basic  component  of  nonlinear  programs  and  n-person  games 
(see  Cottle  (1964),  (1967),  (1968a, b,c),  (1970),  (1971a, b,c). 

Eaves  (1971a, b),  and  Lemke  (1964),  (1965)).  In  this  area  the  research 
program  could  concentrate  on  the  development  and  testing  of  methods 
which  exploit  the  special  structure  of  quadratic  programs,  especially 
ones  of  the  large  size  and  structure  arising  in  major-system 
optimization  problems  [Beale  (1967)]. 

[2]  Nonl inear  Complementarity.  Nonl inear  complementarity  problems 
(see  Cottle  (1966),  Eaves  ( 1 971 d) ,  Karamardian  (1966),  (1971)  and 

Lemke  (1970))  arise  in  general  nonlinear  programs  and  n-person  games 
(n  >_  3).  Normally  such  problems  are  most  efficiently  handled  via 
linear  or  quadratic  approximations.  However,  there  is  a  variety  of 
important  nonlinear  problems  arising  in  practice  whose  special  structure 
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can  be  exploited  to  obtain  more  efficient  procedures.  The  principal 
devices  here  are  (a)  the  use  of  duality  theory  to  obtain  simpler  dual 
problems  or  to  pre-optimize  subproblems  of  a  larger  system,  and  (b)  the 
design  of  special  complementarity  algorithms  which  take  advantage  of 
the  special  structure.  Both  of  these  approaches  could  be  pursued  in 
the  research  program.  A  major  class  of  practical  problems  which  would 
be  investigated  are  the  pooling  or  the  pre-processor  problems.  A  major 
goal  would  be  to  convert  systems  of  allocation  and  pooling  problems 
into  i  -ju i val ent  systems  all  of  one  type  or  the  other. 

[3]  Computation  of  Equilibria  and  Fixed-Points.  One  of  the 
major  outgrowths  of  complementarity  methods  has  been  the  development 
for  the  first  time  of  practical  numerical  methods  for  the  computation 
of  systems  equilibria  and  fixed-points  of  mappings.  (See  Eaves  (1970), 

(1971c, e,f) ,  (1972),  Freidenfelds  (1971),  Kuhn  (1968),  Scarf  (1967a, b,), 

(1969),  (1972),  Rosenmuller  (1971),  and  Wilson  (1971),  (1972).)  The 
advent  of  these  methods  opens  the  possibility  of  computing  directly  the 
equilibria  of  chemical,  biological,  and  physical  systems,  and  equilibria 
n-person  games,  rather  than  via  the  awkward  approximation  methods 
normally  used.  Moreover,  it  raises  the  possibility  of  a  unified  body 
of  theory  and  computational  methods  (since,  for  example,  convex  programming 
problems  can  be  shown  to  be  equivalent  to  finding  the  fixed  points  of 
certain  related  mappings,  and  system  equilibria  are  normally  characterized 
either  via  the  fixed-points  of  the  equations  of  disequilibria  or  in 
terms  of  minimizing  a  measure  of  the  loss  from  disequilibrium).  The 
research  program  could  pursue  the  further  development  of  complementarity 
methods  (including  methods  based  on  primitive  sets  and  simplical 
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subdi vidlons )  for  such  problems  with  particular  emphasis  on  the  development 
of  practical  methods  for  computing  the  equilibria  of  large  systems. 

D.  Combinatorial  Problems  and  Integer  Programing  with  Special 
Structure.  The  fundamental  feature  of  many  systems  optimization  problems 
is  its  combinatorial  character.  This  may  occur  either  because  the 
problem  has  a  special  network  structure  or  because  it  has  discrete 
decision  variables,  so  that  a  huge  number  of  combinations  must  be 
considered.  As  Fulkerson  (1966)  discusses,  such  combinatorial  problems 
arise  in  a  wide  variety  of  contexts.  These  problems  sometimes  can  be 
solved,  of  course,  but  only  by  developing  clever  algorithms  which 
exploit  their  special  structure.  Therefore,  algorithmic  development 
in  this  area  will  be  one  of  the  major  research  activities  of  a  System 
Optimization  Laboratory. 

Probably  the  most  important  combinatorial  problem  for  systems 
optimization  is  the  integer  programming  problem  [Gomory  (1963)].  One 
reason  is  that  these  are  linear  programs  whose  decision  variables 
make  sense  only  with  integer  values  (e.g.,  see  Cushing  (1969)).  In 
addition,  it  is  possible  to  reformulate  a  number  of  important  but 
difficult  (indeed  seemingly  impossible)  problems  of  a  nonlinear,  non- 
convex  and  combinatorial  character  as  mixed  integer  linear  pro¬ 
gramming  problems  (see  Dantzig  (1963)).  Another  important  reason 
is  that  many  large-scale  mathematical  programming  systems  include 
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subproblems  which  are  integer  programs,  so  that  decomposition  methods 
for  such  systems  (e.g.,  see  Bender  (1962))  would  need  an  integer 
programming  algorithm  as  a  subroutine. 

Because  of  these  considerations,  particular  emphasis  will 
be  placed  on  algorithmic  development  for  integer  programming.  This 
has  been  an  area  of  substantial  research  for  over  a  decade,  and  significant 
progress  is  being  made  (e.g.,  see  Hill i er  (1969a),  Balinski  and 
Spielberg  (1969),  and  Geoffrion  and  Marsten  (1972)).  Unfortunately, 
the  problem  is  very  difficult,  and  the  efficiency  of  the  available 
algorithms  does  not  remotely  approach  that  of  the  simplex  method  for 
the  linear  programing  problem.  Therefore,  the  main  thrust  of  this 
research  could  be  the  development  of  special -purpose  algorithms  for 
important  classes  of  integer  programming  problems  which 
exploit  special  structure.  Thorough  testing  and  evaluation  could  be 
conducted,  which  would  necessitate  a  major  programming  effort,  so  the 
resources  and  long-range  continuity  of  the  Systems  Optimization 
Laboratory  would  play  a  vital  role  in  carrying  out  this  development 
beyond  an  initial  stage.  Decomposition  methods  for  mixed  integer  pro¬ 
gramming  systems  could  also  be  investigated.,.  Another  part  of  this 
research  program  would  involve  developing  special-purpose  heuristic 
procedures  (see  Hillier  (1969b)) for  obtaining  good  approximate  solutions 
for  large-scale  integer  programming  systems  having  various  common 
kinds  of  special  structure. 
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E.  Further  Development  of  a  Macro- Language  for  Mathematical 
Programming.  Commerical  codes  for  solving  mathematical  programing 
problems  typically  involve  about  200,000  assembly-language  instructions. 
One  can  anticipate  that  efficient  corrmercial  programs  for  solving 
structured  systems  optimization  problems  will  be  an  order  of  magnitude 
more  complex.  In  order  for  such  programs  to  be  developed  and  maintained, 
the  language  in  which  they  are  written  must  be  highly  readable  and 
easy  to  modify.  This  is  the  purpose  of  the  new  MPL  [Mathematical 
Programming  Language]  now  under  development.  The  continuation  of 
this  work  could  be  one  of  the  major  projects  of  a  Laboratory. 

MPL  is  a  high-level  user-oriented  programming  language 
intended  particularly  for  developing,  testing,  and  communicating 
mathematical  algorithms  (see  Dantzig,  et  al.  (1970)).  It  is  being 
developed  to  provide  a  language  for  mathematical 
algorithms  that  will  be  easier  to  write,  read,  and  modify 
that  currently  available  computer  languages  such  as  FORTRAN, 

ALGOL,  PL/1,  APL. 

The  need  for  a  highly  readable  mathematically-based  computer 
language  has  been  apparent  for  some  time.  Generally  speaking,  standard 
mathematical  notation  in  a  suitably  algorithm-like  structure  appears 
best  for  this  purpose,  since  most  researchers  are  familiar  with  the 
language  of  mathematics.  Therefore,  MPL  closely  parallels  the 
vernacular  of  applied  mathematics.  An  important  area  of  application 
of  MPL  is  for  the  development  and  testing  of  algorithms  for  systems 
optimization  problems.  To  date,  many  methods  have  been  proposed  for 
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solving  such  problems,  but  few  have  been  experimentally  tested  and 
compared  because  of  the  high  cost  and  the  long  time  it  takes  to 
program  them,  and  because  it  is  difficult  to  debug  and  to  modify 
them  quickly  after  they  are  written.  It  is  believed  that  highly 
readable  programs  would  greatly  facilitate  experimentation  with  these 
proposed  methods  and  would  shorten  the  time  until  they  can  be  used 
in  practice.  Thus,  the  development  of  a  sophisticated  version  of  MPL 
would  provide  a  vital  tool  for  the  Systems  Optimization  Laboratory,  as 
well  as  for  other  researchers. 

As  pointed  out  by  William  Orchard-Hays  many  other  special 
purpose  languages  beside  MPL  would  be  required  as  basic  research  tools. 
There  is  a  need  to  have  special  language  for  Job  Control,  Computer 
Control,  Matrix  Generation,  Procedure  Programming  (e.g.,  MPL  or  APL) ; 
languages  for  File  Mechanisms;  languages  for  organizing  the  entire 
system  of  computation. 


To  sunmarize:  Large-Scale  Optimization  requires  laboratories 
where  a  large  number  of  test  models,  computer  programs,  and  special 
"tools"  to  aid  in  developing  variants  of  existing  techniques,  are 
assembled  in  a  systematic  way.  Only  this  way  can  one  hope  to  model 
and  solve  the  host  of  pressing  total  system  problems  that  the  world 
faces  today. 
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